feat: Log the backup stats summary to a file #128
Conversation
The log records the added, removed and snapshot size after each backup.
There was a problem hiding this comment.
Great thank you @gerardbosch, and for using functions!
Juat one comment
etc/restic/_global.env.sh
Outdated
| # (optional) Desktop notifications. See restic_backup.sh for details on how to set this up. | ||
| export RESTIC_NOTIFY_BACKUP_STATS=false | ||
| # Backup summary stats log: snapshot size, etc. (empty/unset won't log) | ||
| export RESTIC_BACKUP_STATS_DIR="$INSTALL_PREFIX/var/log/restic-automatic-backup-scheduler" |
There was a problem hiding this comment.
Could we make this opt-in please, to make it as simple as possible for new users. Advanced users can enable it. We can include an out-commented line as an example of how to enable it.
There was a problem hiding this comment.
Hi! sure!! But just want to double-check it :)
Do you think it is better to have the log disabled by default? Probably I'm biased and looking from my use-case perspective only, but I guessed that would be a sane default to have a summary log enabled by default for all users, and make it an opt-out. For myself I find to log this info an essential feature.
Maybe looking from my limited use case here, hehe (I'm just doing a daily backup of home). The only cons I see of having it enabled by default is that it needs to run restic snapshots and restic diff (I stated it on the README 🙂), which takes a little more time to finish the systemd service (not the backup) and I guess requires extra talk to the remote repo (but as it runs unattended on background I don't care). The benefit is that you can quickly check how the backup increased on every snapshot without ceremony, and helps to investigate the backup.
Let me know, and if you still think it is better to be an opt-in I could just comment # this line and add the (optional) comment 😃, leaving the example directory as-is /var/log/restic-auto... but commented + update the README accordingly. Cheers!
There was a problem hiding this comment.
P.S. I'm not sure if the data transfer supposedly required for restic diff —which is used (single time) for both, log and notification— may be a problem for general public, I don't actually know how it works, or if it computes against a local cache or something like this, couldn't find it in the docs https://restic.readthedocs.io/en/latest/040_backup.html#comparing-snapshots
There was a problem hiding this comment.
Hi, I see your point that it makes sense to have. I personally would like it. Though the project scope has grown a bit too much making maintenance harder (I have much less time at the moment for it). Thus I would much appreciate if this was opt-in. 🙂
There was a problem hiding this comment.
Yep, no problem! I disabled it (left it commented), you can check now ccdf7b7
I was missing to be able to track how the backup repo grows, and finally came up with a log to persist what desktop notification shows after the backup (many times I miss the notification and don't realized if I unconsciously added heavy files to the home backup).
When something gets added unintentionally to the backup I usually check/update the
~/.backup_exclude.txtand manually runsudo resticw forget --prune <offending snapshot ID>. The stats log will help to better track these situations.The stats log feature could be disabled (opt-out) by setting empty the new environment variable
$RESTIC_BACKUP_STATS_DIR.The log records the added, removed and snapshot size after each backup to a CSV log. This is how it looks:
Why a CSV? Just to avoid to fill a file with the same text message on each line. I thought on simply adding a header and just write the data, not the full message each time. New records are prepended in the log, the latest backup sits below the header (descending order), easy to inspect.
Organized with yearly file, e.g.
2023-stats.log.csv